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DETECTION AND PROCESSING OF ANNOTATED ANCHORS 

INVENTORS: 

Gene Golovchinsky 
Catherine Claire Marshall 
Morgan N. Price 
William N. Schilit 

Field of the Invention 

[0001] The invention, roughly described, is directed to a system for 

recognizing annotated anchors in documents, and subsequently processing the 
annotated anchors. 

BACKGROUND 

[0002] Active reading of conventional, paper documents often involves 

marking or annotating text by a reader as they read using a number of methods, the 
most obvious of which is to highlight the text to or place a written notation 
adjacent to the text. Typical annotations include margin notes, vertical bars, 
graphic notations, highlights, underlines, circles, etc. References to external 
documents (ex: hypertext documents, footnotes, etc.) in text generally stand out 
due to their grammatical citation which creates a visual and contextual distinction 
from the surrounding text. 

[0003] There are several currently existing techniques available to users for 

annotating documents. For example, methods exist for annotating online 
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documents using a web browsing facility, electronic meeting systems (e.g. the 
"whiteboard" feature in Microsoft's Netmeeting™), and through word processing 
software. In some of the more basic systems, annotation of documents may 
comprise manually listing the documents (by writing on paper, using the cut and 
5 paste feature of an operating system clipboard, or using the book marking function 
of the browser), none of which leaves a persistent trace within the displayed 
document. 

[0004] More advanced annotation techniques exist for digitally marking 

documents, such as the method of digital ink as described in Schilit et al., Beyond 

10 Paper: Supporting Active Reading with Free Form Digital Ink Annotations, 
Proceedings of CHI98. These more advanced systems reduce the amount of reader 
interruption which often results from basic annotation techniques. 
[0005] Systems also exist for detecting the annotation context and 

organizing those annotations for the user by displaying a list of just the annotated 

15 passages. For example, in United States Patent 6,279,014, filed September 15, 
1997, a technique for organizing annotations in the context of the document in 
which they appear is described. Other systems, such as the one described in United 
States Patent Application Serial No. 09/222,962 filed December 30, 1998, provide 
a method for maintaining free form ink annotations on changing views. This 

2 o technique involves maintaining freeform ink annotations on changing views by 
anchoring ink strokes to specific objects that can move around or change size. 
[0006] Each of these techniques reduces reading interruption and provides 

readers with an overview of the annotated portions of a document. However, in 
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electronic documents, users are often presented with the decision of whether to 
follow a link contained within the document or to continue reading. If a reader 
decides to follow the link, the current document is often replaced by the target 
document, thus interrupting reading flow. Alternatively, if the reader decides not 
to follow the link and continues reading, the reader may forget to return and follow 
the link, may be unable to remember why the link was interesting, or may not be 
able to locate the link. 

[0007] Many strategies are used by readers presented with such decisions. 

One is the "read first, follow links later" strategy. Under this strategy a reader will 
read the entire document and then return and follow the links they found 
interesting. This strategy relies on a reader's ability to remember which links 
were interesting, and where in the document they were located. 
[0008] Another strategy is to "follow links as they are encountered." Using 

this strategy a user follows links they find interesting as they approach each link. 
In addition to interrupting the flow of reading, another problem with this strategy is 
that it may result in a reader losing track of the original document. Often when a 
link is followed the original document is removed from the viewing area and 
replaced with the target of the link. 

[0009] Another strategy for managing the task of reading documents 

containing links is to gather the links using a link-gathering interface. Examples of 
conventional link gathering interfaces include the bookmark facility of Web 
browsers, and spatial hypertexts such as VIKI (Marshall, C.C., Shipman, F.M. Ill 
and Coombs, J (1994) VIKI: Spatial Hypertext Supporting Emergent Structure. In 



Proceedings of The European Conference on Hypertext 1994(Edinburgh, Scotland 
September 19-23): ACM Press, 13-23.); Data Mountain (Robertson, G.G. (1998) 
Data Mountain: Using Spatial Memory for Document Management. In Proceedings 
of UIST98 (San Francisco, CA November 2-4), ACM Press, 153-162.); Web 
Squirrel (Bernstein, 1998, http://www.eastgate.com/squirrel/); and Web Forager 
(Card, S.K., Robertson, G.G., and York, W. (1996) The WebBook and the Web 
Forager: An Information Workspace for the World-Wide Web. In Proceedings of 
CHI96 (Vancouver, BC April 13-18), ACM Press, 11-117.). While this type of 
explicit link gathering may be less computationally intensive than annotation 
processing, it is more difficult on the user for whom annotation is a generally 
intuitive and well-honed skill. 

[0010] Still other systems maintain annotations on Web pages that may 

change unpredictably, but such systems have not considered processing the 
annotations that are near anchors. For example, Equill visual markup, 
(www.equill.com), and I-markup (www.imarkup.com) associate freeform ink 
annotations with objects in a Web page, but do not allow a reader to manipulate 
documents based on these marks. 

[0011] Additionally, a number of systems that provide hypertext graph 

searching capability have been described in literature (e.g., Consens, M.P., Eigler, 
F. Ch., Hasan, M.Z., Mendelzon, A.O. , Noik, E.G., Ryman, A.G., and Vista, D. 
(1994) Architecture and Applications of the Hy+ Visualization System, IBM 
Systems Journal, 33 (3), 458-476). Such tools allow the user to pose queries and to 
manipulate documents based on hypertext topology to identify desired nodes based 
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on graph connectivity. 

[0012] Even with all these different techniques and strategies for making 

reading and annotating documents easier and more manageable, current techniques 
still either interrupt the reader ' s concentration or rely on the reader' s ability to later 
locate the interesting links and remember why they were interesting. 
[0013] Thus, there is a desire for a system which will allow a user to defer 

link following while not losing track of the links location within a document. 
Additionally, there is a desire for a system which extends beyond just explicit link 
gathering interfaces. 

SUMMARY 

[0014] The present invention provides readers with a mechanism that 

allows them to defer link following in a document by detecting the reader's interest 
in a particular external document through their freeform markings and annotations, 
and processing these detected marks in various ways. In general, two different 
kinds of processing are supported: (1) using the reader's marks on or near hypertext 
link anchors - annotated anchors - to define a collection of documents of interest 
from all possible external document references that the reader encounters; and (2) 
using these marks to help the reader visualize the interconnections of interest. 
[001 5] The invention, roughly described, comprises a system for detecting 

annotated anchors in a document. In one aspect, the invention includes a method 
for detecting a reader's annotations in a document and determining the presence of 
a document reference in the document proximate to the annotation. These 



document references are referred to herein as "anchors" and hence the system 
identifies "annotated anchors" in the document under consideration by the reader. 
[0016] In a further aspect, the detection is accomplished by a method, 

which maybe in the form of programming code, which groups annotation marks 
5 and determines whether a document reference is positioned in or near the 
annotations. The system detects both explicit anchors, such as hypertext links, and 
implicit anchors, such as bibliographic references or citations. 
[001 7] In another aspect, the annotated anchors detected are referenced in a 

data structure to allow for further processing of the annotated anchors. In an 

10 embodiment, the data structure may be hierarchical, spatial, sequential, etc. 

[0018] An additional component of the system of the invention is the 

processing which may occur on the annotated anchors once detected. Such 
processing includes: generating a list illustrating the documents that are targets of 
annotated anchors, with or without the annotation metadata, to allow for deferred 

15 link following; constraining the list illustrating the target documents; searching a 
document database for documents containing one or more references to these target 
documents; pre-fetching documents from a document database; allowing 
propagation of the annotations to other anchors which reference the same target in 
the same document or in other documents and/or hypertext structures; and 

2 o suppressing or enhancing links based on defined annotation structures. The list 
may be generated from any serializable structure, such as hierarchical, spatial, 
sequential, etc. 

[001 9] In another aspect, a method for detecting and processing annotated 
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anchors in a document is provided. The method includes the steps of detecting an 
annotation in the document, detecting, proximate to or within the annotation, the 
presence of an anchor; and, processing the annotated anchor. 
[0020] According to still another aspect, an apparatus for detecting and 

processing an annotated anchor in a document is provided. The apparatus includes 
a processor, and a processor readable storage medium in communication with the 
processor which contains processor readable program code. The processor 
readable program code programs the apparatus to detect an annotation in the 
document, and detect, proximate to or within the annotation, the presence of an 
anchor. In addition to detecting annotated anchors, the apparatus is also 
programmed to perform various processes on the annotated anchor. 

BRIEF DESCRIPTION OF THE DRAWINGS 
[0021] The invention will be described with respect to particular 

embodiments thereof. Other objects, features, and advantages of the invention will 
become apparent with reference to the specification and drawings in which: 
[0022] Figure 1 shows various examples of annotations which may be 

formed using conventional annotation techniques; 

[0023] Figure 2 is a block diagram of a hardware system suitable for 

implementing various embodiments of the present invention; 

[0024] Figure 3 is a flowchart outlining a method for detecting and 

processing an annotated anchor, according to an embodiment of the present 

invention; 
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[0025] Figure 4 is a block diagram illustrating examples of various 

processing one might perform on the data retrieved and stored in accordance with 
embodiments of the present invention; 

[0026] Figure 5 is a flowchart outlining the list process, according to an 

5 embodiment of the present invention; 

[0027] Figures 6A and 6B illustrate examples of output generated using the 

list process, according to an embodiment of the present invention; 
[0028] Figure 7 is a flowchart outlining the process of constrain searching, 

according to an embodiment of the present invention; 

1 o [0029] Figure 8 is a flowchart outlining the process of identifying review 

articles or hub documents, according to an embodiment of the present invention; 
[0030] Figure 9 is a flowchart outlining the process of pre-fetching 

documents, according to an embodiment of the present invention; 
[0031] Figure 10 is a flowchart outlining the process of propagating 

15 annotated anchors, according to an embodiment of the present invention; 

[0032] Figure 1 1 illustrates an example of an output from the process of 

propagating annotated anchors, according to an embodiment of the present 
invention; 

[0033] Figure 12 is a flowchart outlining the process of link suppression, 

2 o according to an embodiment of the present invention; 

[0034] Figure 13 is a flow chart outlining the process of propagating 

annotations to hypertext structures, according to an embodiment of the present 
invention; 
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[0035] Figures 14-15 illustrate examples of output resulting from the 

process of propagating annotations to hypertext structures, according to an 
embodiment of the present invention; 

[0036] Figure 16 is a flowchart outlining the process of displaying 

annotated anchors together with a target, according to an embodiment of the 
present invention; and, 

[0037] Figure 1 7 illustrates an example of output resulting from the process 

of displaying annotated anchors together with a target, according to an embodiment 
of the present invention. 

DETAILED DESCRIPTION 
[0038] Embodiments of the present invention provide a system, method, 

and article of manufacture for detecting and processing annotated anchors within 
documents. The system may be applied in any number of physical contexts. For 
example, embodiments of the present invention may be implemented as their own 
application, or work within existing applications, such as Microsoft Word, 
Microsoft Internet Explorer, Netscape Navigator or any other document viewing 
application. Additionally, the system may be implemented to function with a 
single document or a collection of multiple documents. 

[0039] A "document" as described herein, may be any representation of 

thoughts by means of conventional marks or symbols. For example, a document 
may be an electronic document, such as a .pdf, .doc, .wpd, .rtf, html, or any other 
type of similar electronic file. Additionally, a document maybe video, still image, 



or picture. In general, a document may be any graphical representation which is 
viewable by a reader. 

[0040] "Annotation" as described herein, maybe any freeform marking or 

indication which is associated (such as sequentially, logically, etc.) to at least 
portion of a document. An annotation may be positioned anywhere, irrespective of 
document structure, and may be generated with any type of marking, regardless of 
extent or type. For example, an annotation may be a pinpoint, circle, star, 
underline, highlight, square, doodle, written comments, or any other type of 
graphical representation. 

[0041 ] An annotation may be as simple as the placement of a single dot or 

point, as long as the reader is free to position the dot or point. While an annotation, 
such as a dot or point is viewable with the document, the annotation maybe stored 
apart from the document with which the annotation is associated. Additionally, as 
described herein, an annotation is distinguishable from bookmarks, such as those 
used in Microsoft Reader®. Such bookmarks generally cannot be placed 
anywhere on a displayed document, nor can they be made with any type of 
marking. 

[0042] An "anchor" as described herein, is any perceivable manifestation 

of a link or reference to another document or to another portion of the current 
document. An anchor may be explicit, such as a hyperlink in a web page, or it may 
be implicitly defined, such as by the grammar one uses to identify documents in 
certain forms or citations. For example, "http://www.ibm.com/sfasp/" 603 
illustrated in Figure 6A, is an example of an explicit anchor. Alternatively, 
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"Chandler v. Miller" (Fig. 6B), which is a reference to another document, is an 
example of an implicit anchor. 

[0043] As described herein, annotating an anchor is distinguishable from 

selecting an anchor. Readers select an anchor explicitly to perform an action at 
that particular moment, such as accessing a related document or location. 
Annotating an anchor is part of a delayed engagement with the target document. 
By annotating an anchor a reader can continue reading the document without 
having to access the related document or location at that particular moment. 
[0044] Figure 1 represents various examples of annotations 1 00- 1 06 which 

may be generated by a reader on document 1 00 using any conventional annotation 
technique. For example, annotations 101, 102, 105, and 106 are generated by a 
reader digitally highlighting a portion of document 100. Annotations 1 03 , and 104 
are generated by a reader marking on document 100 using any pointing device 
which is capable of accessing any portion of document 100. In general, an 
annotation may be made using any document annotation technique. 
[0045] Figure 2 illustrates a computing device architecture 200 suitable for 

implementing various embodiments of the present invention. The computing 
device architecture 200 includes a processor 220 communicating with a display 
240, a storage device 260, and an input/output interface 225. The storage device 
260 stores a document 222 which is displayable on display 240. The input/output 
interface 225 communicates with any number of conventional input/output devices 
224 such as a mouse 226, a keyboard 228 and/or apen-based device 230. A reader 
manipulates the input/output devices 224 to annotate the document 222 while it is 



displayed on display 240. The computing device 200 then stores these annotations 
in either storage device 260 or a different storage device (not shown). 
[0046] As shown in Figure 2, computing device architecture 200 may be 

implemented using a programmed general purpose computer. However, 
computing device 200 may also be implemented using a special purpose computer, 
laptop computer, any paper user interface coupled to an entertainment context, (i.e. 
video), a programmed microprocessor or microcontroller and any necessary 
peripheral integrated circuit elements, an ASIC or other integrated circuit, a 
hardwired electronic or logic circuit such as a discrete element circuit, a 
programmable logic device such as a PLD, PLA, FPGA, PAL, or the like. In 
general, any device on which a finite state machine capable of implementing 
embodiments of the present invention can be used to implement the computing 
device architecture 200. 

[0047] Additionally, as shown in Figure 2, memory 260 may be an article 

of manufacture, such as a computer readable medium. For example, memory 260 
may be a magnetic hard disk, an optical disk, a floppy disk, CD-ROM (Compact 
Disk Read-Only memory), RAM (Random Access Memory), ROM (Read- Only 
Memory), or any other readable or writeable data storage technology, singly or in 
combination. 

[0048] Memory 260 may include an operating system 205, such as 

Microsoft Windows® or Unix®, wherein the operating system 205 is capable of 
executing programs or applications using the computing device architecture 200. 
An embodiment of the present invention is implemented as a software program 
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207, and is stored on memory 260. In an embodiment, software program 207 can 
be implemented to function as its own application or may be implemented to 
function within an existing application, such as Microsoft's Internet Explorer, 
Netscape Navigator, Microsoft Word, or any other type of document viewing 
application. Additionally, embodiments of the present invention may be 
implemented to function with any type of language, such as English, Japanese, 
Spanish, etc. 

[0049] As will be understood, embodiments of the present invention, such 

as software program 207, may be in the form of a software program, a software 
object, a software function, a software subroutine, a software method, a software 
instance, a code fragment, a hardware operation or user operation, singly or in 
combination. It is understood that software program 207 can be implemented 
using any programming code, such as C, C++, FORTRAN, Visual Basic, Java, etc. 
Additionally, software program 207 may be implemented using one, two, or any 
number of computing devices 200. 

[0050] Furthermore, it should be appreciated that the link 270 connecting 

the memory 260 and the processor 200 can be a wired or wireless link to a network 
(not shown). The network can be a local area network, a wide area network, an 
Intranet, the Internet or any other distributed processing and storage network. 
[0051] Figure 3 is a flowchart outlining a method for detecting an 

annotated anchor within a document, according to an embodiment of the present 
invention. As one who is skilled in the art will appreciate, Figures 3, 5, 7-10, 12, 
13, and 16 illustrate logic boxes for performing specific functions. In an alternate 
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embodiment, more or fewer logic boxes may be used. In an embodiment of the 
present invention, a logic box may represent a portion of software program 107 
illustrated in Figure 2, or any portion of embodiments of the present invention. 
[0052] The method begins at logic box 301, where the system receives a 

document. In logic box 302 a reader begins annotating the document. Annotations 
can be made within a document using any currently available document annotation 
technique. As a reader is reading the document and making annotations, control is 
passed to logic box 303. 

[0053] In logic box 303 the system determines whether there is an 

annotation within the document. If it is determined that there is no annotation 
within the document control is returned to logic box 302 and the process is 
repeated until the document is removed or an annotation is made. If however, it is 
determined in logic box 303 that an annotation has been made, control is passed to 
logic box 305. 

[0054] In logic box 305 it is determined whether an anchor is proximate or 

within the annotation. If it is determined in logic box 305 that there are no anchors 
proximate or within the annotation, control is passed to logic box 306, and the 
annotation is ignored in the process, or manipulated according to other current 
annotation processing schemes. Once the annotation has been ignored or otherwise 
processed, control is returned to logic box 303 and the process is repeated. 
[0055] If however, it is determined in logic box 3 05 that there is an anchor 

proximate or within the annotation, control is passed to logic box 307 and the 
anchor located in logic box 305 is added to a list of annotated anchors and control 
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is passed to logic box 309. 

[0056] In logic box 309 the list of annotated anchors generated in logic box 

307 is processed. The list may be processed using any of the processing 
techniques illustrated in Figure 4 and described below, singly or in combination. 
5 Once the desired processing has been performed in logic box 309, control is 
returned to logic box 303 and the logic boxes are repeated until the document is 
removed or all annotations have been processed. 

[0057] In an embodiment, other techniques for modifying or managing 

annotations may be performed prior, subsequent to, or during the process 300. For 

10 example, in an embodiment, annotations made in logic box 302 may be modified 
according to the techniques described in U.S. Patent No. 5,889,523 titled "Method 
and Apparatus for Dynamically Grouping a Plurality of Graphic Obj ects," which is 
herein incorporated by reference, which group a plurality of objects, or 
annotations, based on time and space. Additional techniques for grouping and 

15 modifying annotations may include adding to an annotation made by a reader the 
remainder of a sentence, or paragraph, if a portion of the sentence or paragraph was 
not originally annotated. 

[0058] In such an example, modifications may be made prior to the 

processing of logic boxes 303-309, and logic boxes 303-309, described above, will 
20 be processed using the modified annotations. 

[0059] Additionally, in an embodiment logic boxes may be removed from 

the process illustrated in Figure 3. For example, the process may be performed 
without adding the annotated anchor to an annotated anchor list 307. In such an 
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embodiment, once an annotated anchor is determined in logic box 305, control is 
passed to logic box 309 and the anchor is processed according to any of the 
processing techniques described below. 

[0060] Figure 4 illustrates the different processes which may be performed 

on the list of annotated anchors generated using the process described with respect 
to Figure 3, according to an embodiment of the present invention. Annotated 
anchor data 402 relates to the list of annotated anchors generated in logic box 307 
during the process 300 (Figure 3). It will be understood that the processes 
illustrated in Figure 4 may be performed on the list of annotated anchors 402 or 
alternatively may be performed on each annotation as it is made in a document. 
[0061 ] Once a reader has finished reading and annotating a document they 

may wish to follow the links which were annotated during their reading. 
Accordingly, the list process 404 will be performed on the annotated anchor data 
402 to generate a "to-read" list. List process 404 allows a reader to read a 
document, annotate any portion of the document using existing annotation 
techniques, and later review a list of only annotations containing anchors. The 
system provides a reader with the ability to keep track of anchors which they find 
interesting, without interrupting reading. The user can annotate on or around the 
anchor, and defer the accessing of the related document until a later time. 
[0062] Figure 5 illustrates a flowchart outlining the list process 404 

illustrated in Figure 4. Upon initiation of the process in logic box 501, control is 
passed to logic box 503 where the system obtains annotated anchor data 402. 
[0063] In logic box 505 the system determines the readers preferences for 



displaying the annotated anchor data 402. For example, the user may desire to 
organize the list based on metadata of the target documents (such as the target 
document's title, author, size, when created, etc.), metadata of the annotations 
(such as the location of the annotations within the present document, when the 
annotations were made, the color, shape, size of the annotations, etc.), metadata of 
the annotated anchor (such as the link it represents, its location in the document, 
etc.), or any other desired organizational technique. Additionally, a reader may 
desire to view the entire annotation which contains each anchor, just the first 
annotation containing each anchor, just the anchors themselves, or any other 
display format selected by the reader. 

[0064] In logic box 507 a "to-read" list is generated from the annotated 

anchor data 402 based on the readers preferences obtained in logic box 505 . Upon 
generation of a "to-read" list, control is passed to logic box 509 and the "to-read" 
list is displayed on display 240 (Fig. 2). 

[0065] Figures 6A and 6B illustrate examples of "to-read" lists 600, 650 

generated from the anchor data 402 using the list process 404, according to an 
embodiment of the present invention. The lists 600, 650 may be generated from 
one, two, or as many documents as selected by the reader. 
[0066] Figure 6 A includes anchors 601, 603, 605, 607, annotations 609, 

611, 613, 615, 617, 617 and some surrounding context 621, 623, 625, 627, 629 to 
make the anchors more comprehensible. 

[0067] The list may be formatted according to any user created attributes 

and may include annotations from one, two, or as many documents as desired. For 



example, list 600 is formatted based on the order in which the annotated anchors 
601-607 first appear in the document titled "Yeung et al., 1998." Additionally, a 
reader may select to group annotations containing anchors which point to the same 
target. For example, in Figure 6 A annotations 609 and 611 both include an anchor 
which points to the same target document. Thus, annotations 609 and 611 may be 
grouped and listed under the same anchor 601 . 

[0068] Figure 6B illustrates another "to-read" list 650 of annotated anchors 

generated using the list process 404 illustrated in Figure 5, according to an 
embodiment of the present invention. The list 650 is formatted first by target 
documents of the annotated anchors and ordered by frequency of annotations. In 
this example, the target document which has corresponding anchors most 
frequently annotated is Vernoniav. Action, Supreme Ct, June 1995 651. The next 
most frequently annotated target document is Skinner v. Railway Labor Executives ' 
Assoc., Sup. Ct, March 1989 652. Included under each target document heading 
651, 652 is the document from which the annotation is located 653, 655, 657, the 
annotated anchor which points to the target document 659, 661, 663, and 
surrounding context 665, 667, 669. 

[0069] Each of the to-read lists 600, 65 0, or any other to-read list generated 

according to the process illustrated in Figure 5, provides a reader access to the 
linked document, access to the context of the annotation, as well as access to the 
document in which the annotation was made. A reader can select a target in the list 
to follow the link, or select the annotation to return to that portion of the document. 
[0070] The annotated anchor data 402 may also be treated as a collection, 



and content and/or metadata searches of the collection may be performed 
according to an embodiment of the present invention. For example, a reader may 
select the constraint search process 406 and have the system display only 
annotations which were made using a particular color, such as yellow, annotations 
of a particular shape, such as a circle, or any other user-defined criteria. 
[0071] Additionally, if a reader wishes to determine the meaning of a 

technical term used within the document they are currently reading, the constraint 
search process 406 will search the target documents which the annotated anchors 
point to and provide an overview of how the term is used in these documents. This 
overview may be displayed in a format similar to that shown in Figures 6A and 6B. 
Alternatively, the results may be ranked or filtered by the number of annotated 
anchors for each returned document, the types of annotations on the anchors, the 
type of the link that was annotated, etc. 

[0072] Figure 7 illustrates a flowchart outlining the process of constrain 

searching 406 of Figure 4. The process begins at logic box 701 and control is 
passed to logic box 703 where the system obtains annotated anchor data 402. 
[0073] In logic box 705 the system determines the constraint criteria 

selected by a reader. For example, the constraint criteria may be the color or shape 
of an anchor, or searching the target documents for a particular term. 
[0074] In logic box 707 the system searches for all information which 

meets the constraint criteria and generates a constrained list. Upon generation of 
the constrained list control is passed to logic box 709 and the list is displayed on 
display 240 (Figure 2). 
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[0075] The process illustrated as block 408 in Figure 4, provides a reader 

with the ability to search for "hubs" that cite many target documents which are 
referenced by an annotated anchor. For example, if a researcher has collected a 
dozen references of interest, and would rather read an article reviewing those 
5 documents than read the dozen papers, the researcher can locate a hub document 
that cites some or all of the other documents. Additionally, an embodiment of the 
present invention can provide an interface for refining such searches, for example 
by marking individual documents in the list as being more or less important. 
[0076] Figure 8 illustrates a flowchart outlining the process of identifying 

1 o review articles or hub documents 408 of Figure 4. The process begins at logic box 

801 and control is passed to logic box 803 where the system obtains annotated 
anchor data 402. 

[0077] In logic box 805 the system locates a document or documents which 

reference at least one target document identified in annotated anchor data 402. The 
1 5 system may search a predefined list of documents for a reference, or alternatively 
the system may search for any document which contains references to multiple 
target documents. Once a list of search results has been generated, control is 
passed to logic box 807 and the system displays the results of the search performed 
in logic box 805 on display 240 (Fig. 2). In an embodiment, the results may be 

2 o displayed in a format similar to that illustrated in Figures 6A and 6B, or in any 

other reader-selected format. 

[0078] The process of pre-fetching documents 410 provides additional 

functionality and portability to embodiments of the present invention. This 



technique is often useful when the system is implemented using a device which 
may not have constant access to target documents. Such devices may be laptops, 
Palm tops, or other mobile computing devices. 

[0079] The process of pre-fetching 410 obtains target documents of 

annotated anchors prior to the time the reader actually attempts to access the 
document. 

[0080] Current techniques for pre-fetching documents obtain all of the 

documents referenced by the document being viewed. This is often undesirable 
because there may be many referenced documents and to pre-fetch all of them 
would be expensive, and both time and space consuming. An example of such as 
system is described in CiteSeer (Giles et al, 1998) which runs searches over the 
Web to identify articles corresponding to references in academic papers. The 
CiteSeer system retrieves each of these articles and builds an article reference 
database. 

[0081] Figure 9 illustrates a flowchart outlining the process of pre-fetching 

documents 410 of Figure 4, according to an embodiment of the present invention. 
The process begins at logic box 901 and control is passed to logic box 903 where 
the system obtains an annotated anchor. The annotated anchor maybe obtained as 
soon as the reader performs the annotation or alternatively from annotated anchor 
data 402. 

[0082] In logic box 905 the system determines whether the target document 

has already been pre-fetched or already contained in memory. If it is determined in 
logic box 905 that the system already has the target document, control is returned 



to logic box 903 and the process continues until all annotated anchors have been 
processed. If however, it is determined that the target document has not been pre- 
fetched, control is passed to logic box 907. 

[0083] In logic box 907 the system fetches the target document and stores it 

in memory 260 (Fig. 1 ) or another memory location which will later be accessible. 
Once the target document has been fetched, control is passed to logic box 909 and 
the system determines whether there is another annotated anchor within the 
document. If it is determined that there is another annotated anchor, control is 
returned to logic box 903 and the process is repeated for that annotated anchor. If, 
however, it is determined in logic box 909 that there are no other annotated 
anchors, control is returned to logic box 309 of Figure 3. 

[0084] In addition to using annotated anchors to generate a "to-read" list, 

and to gather/organize and pre-fetch documents, the process of propagating 
annotations to other anchors 412 illustrated in Figure 4 may be implemented 
according to an embodiment of the present invention. Propagating annotations to 
other anchors helps a reader visualize interconnections of interest, or alternatively 
to suppress undesired links. Visualizing interconnections provides a reader with 
advanced notice as to whether they found an anchor interesting. The technique of 
propagating anchors provides a trail marking facility which reduces the 
disorientation that often accompanies densely anchored documents. By 
propagating annotations of anchors to other anchors which point to the same target, 
a reader can determine whether an anchor points to a target whose anchor has 
already been annotated. This is often beneficial when several different anchors 



reference the same target. 

[0085] Additionally, when a reader sees an annotation which has been 

propagated to a subsequent anchor, the reader can select the annotation and the 
system will display the original annotated anchor or anchors (if the anchor has 
been annotated more than once). Providing this additional information allows a 
reader to select between viewing more context for a specific anchor, navigating to 
the full context of the annotated anchor, navigating to the target of the anchor, or 
returning to reading the original document. 

[0086] Figure 10 illustrates a flowchart outlining the process of 

propagating annotations to other anchors 412 of Figure 4, according to an 
embodiment of the present invention. The process begins at logic box 1001 and 
control is passed to logic box 1 003 where the system obtains an annotated anchor. 
The annotated anchor may be obtained as soon as the reader performs the 
annotation or alternatively, from annotated anchor data 402. 
[0087] In logic box 1 005 the system determines the target of the annotated 

anchor. The target may be another document or another location in the same 
document. Once the target is determined, control is passed to logic box 1007 and a 
determination is made as to whether there is another anchor which points to the 
same target. The system may search for other anchors within the same document, 
or for anchors in any document which may be accessed by the reader. 
Additionally, as new documents are accessed by a reader, the system may scan 
those documents to determine if anchors exist within the document and if the 
anchors point to a target with an already-annotated anchor. If it is determined in 
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logic box 1007 that there are other anchors which point to the same target as the 
annotated anchor, control is passed to logic box 1009. If however, it is determined 
in logic box 1007 that there are no other anchors which point to the same target as 
the annotated anchor, control is passed to logic box 1011. 
[0088] In logic box 1 009 the anchor located in logic box 1 007 is annotated 

with an annotation similar to the annotation made by the reader on the original 
anchor. For example, if a user circles an anchor, and another anchor which points 
to the same target is located, the system will circle the newly-found anchor. 
[0089] In logic box 1011 it is determined whether there is an annotated 

anchor which represents a different target. If an annotated anchor to another target 
exists, control is returned to logic box 1 003 and the process is repeated. However, 
if no other annotated anchors exist, control is returned to logic box 309 of Figure 3. 
[0090] Figure 11 illustrates a snippet of two web pages 1100 and 1105, 

illustrating an example of an output resulting from the process of propagating 
annotations 412, illustrated in Figure 10, according to an embodiment of the 
present invention. This technique propagates annotation 1 102, made by a reader 
on or around anchor 1 1 0 1 , to other anchors 1 1 03 which reference the same target. 
In an embodiment, once an anchor has been annotated in one document, the 
annotation can be propagated to all other anchors pointing to the same target. 
These anchors may be in the same document or in other documents which the 
reader may access. For example, referring to Figure 1 1 , if a reader annotates the 
anchor "People Finder" 1101 with a check annotation 1 102, all other instances of 
an anchor which points to the same target will receive a similar annotation. Thus, 
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"People Finder" 1 103 located in document 1 105 is updated with a similar check 
annotation 1 104. By updating anchors which point to the same target as an 
annotated anchor the reader will know that each of those anchors point to the same 
target. 

5 [0091] hi addition to propagating annotations to other anchors which 

reference the same target, a reader may also select the process of link suppression 
418 (Figure 4) and have all anchors relating to an annotated anchor suppressed. 
For example, when a reader approaches a densely-linked document with a 
particular information need, the reader may wish to suppress some links that are 
10 not relevant to the current situation. If the reader crosses out an anchor to an 
unwanted link, the system can subsequently suppress all anchors which point to the 
same target, thereby reducing visual clutter and eliminating the distraction of 
deciding whether or not to follow, or annotate such links. 

[0092] The process of link suppression 418 illustrated in Figure 12 

15 functions in a similar manner to the process of propagating annotations to other 
anchors 412, illustrated in Figure 10. 

[0093] Figure 12 illustrates a flowchart outlining the process of link 

suppression 418 of Figure 4, according to an embodiment of the present invention. 
The process begins at logic box 1201 and control is passed to logic box 1203 
2 o where the system obtains an indication from a reader that a anchor is undesired, or 
uninteresting. The indication may be made by the reader marking an anchor in a 
particular manner to designate that it is not interesting. Such marking may be in 
the form of striking out the anchor, marking an X on the anchor, or any other 



reader selected marking. It will be appreciated that a reader may make a marking 
on an anchor contained in a document, on an anchor which is being displayed as a 
result of any of the above-described processes, or on any other anchor which is 
viewable by the reader. 

[0094] Once a marked anchor has been received, control is passed to logic 

box 1205 where the system determines the target of the marked anchor. Once the 
target is determined, control is passed to logic box 1207. 

[0095] In logic box 1207 a determination is made as to whether there is 

another anchor which points to the same target. The system may search for other 
anchors within the same document, or for anchors in any document which may be 
accessed by the reader. Additionally, as new documents are accessed, the system 
may scan those documents to determine if any anchors pointing to the target exist 
within the document. If it is determined in logic box 1207 that there are other 
anchors which point to the same target as the marked anchor, control is passed to 
logic box 1209. 

[0096] In logic box 1209 the anchor located in logic box 1207 is 

suppressed using a reader selected suppression technique. For example, a reader 
may desire to have all anchors which relate to an uninteresting target grayed out, 
presented in a format matching the other existing text, or in any other desired 
fashion. Once the anchor located in logic box 1207 has been suppressed, control is 
returned to logic box 1207 and the process is repeated. 

[0097] If however, it is determined in logic box 1 207 that there are no other 

anchors which point to the same target as the annotated anchor, control is passed to 



logic box 1211 and the system determines whether there are any other anchors 
which have been marked by a reader to indicate that they are undesirable. If it is 
determined that there are other undesired anchors, control is returned to logic box 
1205 and the process is repeated for each additional undesired anchor. If, however, 
there are no other undesired anchors, control is returned to logic box 309 of Figure 
3. 

[0098] Annotations of anchors may also be propagated onto visualizations 

of hypertext structures using the process of propagating annotations to hypertext 
structures 414, illustrated in Figure 4. Generating and displaying hypertext 
structures which represent a group of documents and their interconnections is well 
known in the art and will not be described in detail. 

[0099] Hypertext structures illustrating a large number of documents with a 

large number of interconnections are often very dense and difficult to understand. 
The process of propagating annotations to hypertext structures 414, described with 
respect to Figure 1 3 makes these structures more distinctive and easier for a reader 
to understand, as illustrated in Figures 14 and 15. 

[0100] Figure 13 illustrates a flowchart outlining the process of 

propagating annotations to hypertext structures 414 of Figure 4, according to an 
embodiment of the present invention. The process begins at logic box 1301 and 
control is passed to logic box 1303 where the system obtains an annotated anchor. 
[0101] In logic box 1305 the system determines the target of the annotation 

and in logic box 1 307 the system determines a node on a hypertext structure which 
represents the target document. In logic box 1309 the node located in logic box 
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1307 is annotated with an annotation similar to the annotation made by the reader 
on the original anchor. For example, if a user annotates an anchor with a circle, a 
circle will be propagated onto a node which represents the target of the circled 
anchor. 

5 [0102] In logic box 1 3 1 1 the system determines whether there is another 

annotated anchor which has not been propagated. If it is determined that there are 
other annotated anchors, control is returned to logic box 1303 and the process is 
repeated. If however, it is determined that there are no other annotated anchors, 
control is passed to logic box 1313 and the hypertext structure with the propagated 

10 annotations is displayed on display 240 (Figure 2). 

[0103] Figures 14 and 15 illustrate examples of an output resulting from 

the process of propagating annotations to hypertext structures 414 illustrated in 
Figure 13, according to an embodiment of the present invention. 
[0104] In Figure 14 each node in the hypertext structure 1400, such as 

15 nodes 1401 , 1402, and 1403 represent a document. The lines interconnecting each 
node, such as lines 1405, 1407, and 1409 represent a link, either unidirectional or 
bi-directional, between documents. Annotations 1411, 1413, 1415, and 1417 
illustrate annotations which have been propagated to a node 1421, 1423, 1425, 
1427 using the process of propagating annotations to hypertext structures 414 

2 o illustrated in Figure 1 3 . Each annotation 1411-1417 represents an annotation made 
by a reader on an anchor, and each node 1421-1427 represents the target of the 
respective annotated anchor. For example, annotation 1417 represents an 
annotation made on an anchor which points to a document tilted xclipboard, and 
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node 1427 represents target document xclipboard. 

[0105] Figure 15 illustrates another hypertext structure 1500 containing 

propagated annotations, according to an embodiment of the present invention. In 
Figure 1 5 each node in the hypertext structure 1 500, such as nodes 1 501 , 1 502, and 
1 503 represent a document. The link lines interconnecting each node, such as link 
lines 1505, 1507, and 1509 represent a link, either unidirectional or bi-directional, 
between documents. Annotations 1511, and 1513 illustrate annotations which have 
been propagated to a node 1521, and 1523 using the process of propagating 
annotations to hypertext structures 414 illustrated in Figure 13. Each annotation 
1511-1513 represents an annotation made by a reader on an anchor, and each node 
1521-1523 represents the target of the respective annotated anchor. For example, 
annotation 1511 represents an annotation made on an anchor which points to a 
document tilted human factors, and node 1523 represents target document 
electronic mail. 

[0106] Additionally, in an embodiment, the system can display on the 

hypertext structure 1 500 what document the annotation is being propagated from 
by altering the representation of the link line. For example, link line 1531 is 
displayed thicker than the other link lines and with an arrow, thereby indicating 
that annotation 1511 is being propagated from document 1502. Accordingly, 
document 1 502 contains an anchor to document 1521 which has been annotated by 
a reader using a check mark. 

[0107] In an embodiment, a user may select to have annotations made in a 

document propagated to the node representing the document in which the 



annotation is actually contained. For example, referring again to Figure 14, if a 
user made annotation 1 4 1 3 in a document titled pbmtogem. 1 , the annotation would 
be propagated to node 1423 in the hypertext structure 1400 which represents 
document pbmtogem. 1 . 

[01 08] Additionally, in an embodiment, anchors which have been marked 

by a reader as uninteresting may have the node and/or the link line which 
corresponds to the uninteresting anchor and link suppressed in the display of the 
hypertext structure 1400. For example, referring to Figure 15, a user has marked in 
a document represented by node 1541 an anchor which points to a target 
represented by node 1543 as uninteresting. Accordingly, link line 1545 has been 
suppressed and node 1543 has been suppressed 1547. 

[0109] Once a reader has decided to follow an annotated anchor, either 

from a to-read list, from an actual annotation, from a hypertext structure, or from 
any other location, the system will display the target of the annotated anchor and 
either some or all of the annotations which include an anchor to the target. 
[0110] Figure 1 6 illustrates a flowchart outlining the process of displaying 

annotated anchors together with target 416 of Figure 4, according to an 
embodiment of the present invention. The process begins at logic box 1601 and 
control is passed to logic box 1603 where the system receives a selection by a 
reader of an annotated anchor. This selection may be from any of the outputs of 
the above-described processes or directly from a document. 
[0111] In logic box 1605 the target of the selected annotated anchor is 

determined and obtained. In logic box 1607 a determination is made as to whether 
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there are other annotated anchors which point to the same target. If it is 
determined that other annotated anchors point to the same target control is passed 
to logic box 1 609 and each of the other annotated anchors which point to the same 
target are gathered. If however, it is determined that there are no other annotated 
5 anchors which point to the target control is passed to logic box 1611. 

[0112] In logic box 1611 the target and each of the annotated anchors 

which point to the target are displayed on display 240 so that all information is 
simultaneously viewable. 

[0113] Figure 17 illustrates an example of an output generated from the 

10 process of displaying annotated anchors together with target document 416, 

illustrated in Figure 16, according to an embodiment of the present invention. 

Once a reader has selected an anchor, the annotation 1701 which includes the 

selected anchor, any other annotated anchor which points to the same target, and 

the target 1703 are made viewable. By displaying one or more annotated anchors 
15 together with a target, the system reminds the reader of why they decided to read 

the target 1703, and where they have seen references to the target 1703 before. 

The annotated anchors may also help the reader interpret the target. 

[0114] It should be understood that the particular embodiments described 

above are only illustrative of the principles of the present invention, and various 
2 o modifications could be made by those skilled in the art without departing from the 

scope and spirit of the invention. Thus, the scope of the present invention is 

limited only by the claims that follow. 



